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ABSTRACT 

Correlations between the Enroll ee Test Battery (ETB) 
and performance on more conventional tests were examined to determine 
the ETB validity, in addition to a small- Scale study of predictive 
validity of both ETB and the conventional tests. Three types of paper 
and pencil measures were utilized: the ETB, conventional cognitive 
skills tests, and scales of criterion questionnaires. There were 23 
tests in all. This study sample consisted of 74 (29 males and 45 
females) predominantly black high school dropouts, between 14 and 17 
years of age, who were enrolled in two Neighborhood Youth Corps 
Out-of— School projects. Answers to the criterion questionnaires were 
obtained from 44 of the enrollees six months after they had left the 
Predictive validity for the tests was examined by 
correlating each of the 23 tests scores with factor scores derived 
from the questionnaire criterion scales for each of the three 
criterion samples — Program Completion, Post Program (Employed) , and 
Post Program (Not Employed) . Levels of concurrent validity for the 
ETB was determined from intercorrelations between the 17 tests of 
that battery and the 6 conventional cognitive skill measures. That 
matrix was also factor analyzed. Results of the study show that 
measures of the ETB, which was designed specif iced, ly for use with 
disad'vantaged adolescents, appear fairly coherent and logical in 

patterns of relationships with each other and with conventional 
cognitive skills tests. (DB) 
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Validation of a Test Battery for Youth-Work 
Training Program Enrollees^ 

Norman E. Freeberg 
and 

Richard R. Reilly 

Background and Purpose 

A battery of paper and pencil measures, designed for use as guidance 
tools vfith youth-work training program enrollees, had undergone initial 
development on the basis of data obtained from adolescent male and female 
school dropouts in the Neighborhood Youth Corps Program (Preeberg, 1970)* 
Measures comprising that experimental battery (Enrollee Test Battery) vfere 
tailored, in terms of content, verbal level, format and method of administra- 
tion, to overcome a number of deficiencies often attributed to conventional 
formal tests vfhen applied to youngsters from poverty-level backgrounds who 
are also minority group members with low verbal skill levels (Eells et al. , 1951 
Karp & Sigel, 1965j SPSSI, 1964). The particular measiirement constructs chosen 
for incorporation in the Enrollee Test Battery vrere intended to reflect trainee 
behaviors considered relevant to vfork training program p\rrposes and to the 
informational needs of program professionals (e.g., guidance counselors). 

Those constructs are embodied in a variety of attitudinal, practical reasoning 
and vocational orientation measures found in 15 separate test booklets that 
combine pictorial and verbal information. The booklets are relatively short 

^Funds for this study were provided in part by the Nei^borhood Youth 
Corps projects of Mercy-Douglas Hospital, Philadelphia, Pa., and the Greater 
Chester Movement of Chester, Pa. The authors acknowledge the invaluable 
support of the directors of those N.Y.C. projects, Mr. Eben Short and Mr. 

Norman Watts, as v;ell as the assistance of the project counselors. 
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in length (i.e., 10 to 28 items), are presented without time limits and require 
oral presentation by the examiner. 

In the initial phase of development, the suitability of the measures 
was determined by analyses of their item characteristics, estimates of 
internal consistency (reliability), the pattern of relationships between the 
measures of the battery (factor structure) and the validity of each measiire 
against concurrently obtained criterion rating scores from program guidance 
cotu’.selors and work-site supervisors. Although a number of the tests showed 
reasonable promise on the basis of their measurement properties, broader 
determinations of validity, both concurrent and predictive, are clearly in 
order before the measures covild prove professionally defensible for applica- 
Won to disadvantaged adolescent groups. 

The present study is undertaken as a next-step in what should be a con- 
tinuing process of meeting such validation needs by demonstrating test relation 
ships to a variety of relevant performance criteria. A primary intention is 
to examine--for the first time — correlations between the newly-devised Enrollee 
Test Battery and performance on more conventional, formal tests, some of which 
have either bean used extensively with disadvantaged yoimg adults or carry 
explicit assertions, by the publisher, of their applicability to such a 
population. The set of conventional measures chosen for that purpose is 
intended to span a range from verba], and conceptvial abilities (e.g., vocabulary 
arithmetic, reading con5>rehension) to nonverbal reasoning (e.g., figvtre 
analogies) and associative, rote-manory skill. 

In addition to validation based on concurrent test performance, there is 
a limited opportunity to exeunine predictive validity for the experimental test 
battery as well as for the jinblished measures. This is to be done by 
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iitilizing criterion data obtained for relatively small samples of youth v/ork 
training program enrollees at the time they completed their stay in the 
program and again some six months after they had left. 

Because of expected restrictions in sample size, geographic locale and 
the scope of criterion information available, this predictive validity phase 
should be viewed solely as a small-scale, or pilot-study effort. It is 
atten5>ted here in order to provide preliminary results, as well as needed 
experience with the difficult follow-up and data gathering procedures required 
for a sample of minority group, school dropouts residing in inner-city ^etto 
areas. Such information is expected to be of value in planning Itirther 
larger-scale validation studies. 

Method 

j 

Description of the Measures 

Described below are the essential features of the three types of 
paper and pencil measures utilized; the Enrollee Test Battery, the con- 
ventional co{?nitive skill tests and the scales of the criterion questionnaires 

A. i^rollee Test Battery (ETB) . Only a brief description of the ETB, 
designed explicitly for disadvantaged adolescents, is either warranted or 
practical for piu’poses of this paper. More complete descriptions of the 
rationale, design and measurement characteristics are available elsewhere 
(Freeberg, 1968, 1970). 

Of 15 measures originally developed, 11 were considered acceptable, 
by virtue of content and psychometric characteristics, for this validation 
study. Two are measuj-es of practical reasoning skill; 



(1) Practical Reasoning - Zip Coding (lO itenis ) : provides the 

respondent with information needed for a hypothetical post 
office job requiring him to sort mail \ising zip codes. Questions 
are presented in a multiple-choice format pertaining to that task, 

(2) Practical Reasoning - Map Reading (lO items): presents a map 

illustrating several square city blocks along with information 
needed to answer multiple choice questions about delivering 
materials to different locations shown on the map. 

Measures of vocational orientation consist of: 

3. Job Knowledge (27 itenui): requiring answers to multiple- 

choice items regarding a variety of jobs (e.g. , cairpenter, auto 
mechanic, policeman) in terms of their educational requirements, 
starting salary, primary task performed, hours of work, place 
where work is performed and tools utilized. 

(4) Job Seeking Skills (17 items): presents multiple-choice items 

dealing with ways of looking for jobs, some of which entail inter- 
pretation of newspaper want ads and portions of job application 
blanks . 

(5) Job Holding Skills (U items): which depicts situations that 

might be encountered on a job v/ith regard to supervisors' requests, 

. appropriate dress, punctuality, etc., for which the respondent 
chooses the reply he would give, or the most appropriate behavior 
called for, in that situation. 

(6’ Vocational Aspirations minus Plans (VA-VP Discrepancy) (16 
items): is derived from two test booklets, both showing the same 

jobs (pictorially and by designation). The respondent indicates 



the degree to which he would aspire (prefer) to engage in any 
particular occupation and then the degree to which he believes 
it is likely that he would enter that same occupation. 'Che 
discrepancy beWeen scores for aspiration and plans v^as utilized 
for all analyses. 

Attitudinal measures of the battery deal with: 

(7) Attitude Toward Authority (l2 items): in which scenes 

depicting responses of an adolescent to authority figures (e.g., 
teacher, policeman, parents. Judge, etc.) are used to elicit the 
degree of pro- or anti -authority feelings by the respondent. 

(8) Self-Esteem (16 items): depicts situations pictorially in 

v^hich the respondent indicates the degree to which he feels him- 
self "worthy" of the desirable outcomes (getting a diploma, job, 
premotion, etc.). 

(9) Motivation for Vocational Achievement (lO items): presents 

statements (item stems) that bear on the desirability of seeking 
and maintaining employment. These reqiure the individual to respond 
(Likert- Scale format) in terms of the degree of agreement with 

the statement. 

( 3 D) Deferred Gratification (16 items): presents statements dealing 

vfith the v^illingness to delay present reward for future gain to 
which the respondent indicates his degree of agreement. 

A final test booklet in the battery covers: 

(11) Vocational Interest (28 items): which presents pictorial and 

verbal information regarding Job tasks characteristic of a number 
of occupations. The respondent indicates the degree to which he 



could favor performing each task. Separate scale scores are 
derived for each of seven interest categories (If- items each) 
designated as Clerical, Service, Technical, Outdoor, Science, 
Business and Aesthetic. 

B. Formal cognitive skills measures . In addition to the above 
Enrollee Test Battery, six measures of the more conventional, published 
variety of paper and pencil ter.ts served as the concurrent criteria. One 
is the Paragraph Meaning subtest of the Stanford Achievement Test, Form W, 
Intermediate I (Harcourt, Brace, & World, 1964), v^hich has been iised ex- 
tensively for determining reading levels of youngsters from all socioecoronic 
levels and in a variety of educational settings. Two measures, designated 
as the Arithmetic and Vocabulary subtests, were selected from the Adult Basic 
Learning Exam (ABLE), Level II, Form A (Harcourt, Brace, Jovanovich, 1970). 
These ABLE measures appear largely conventional in content and format 
although they are said to be designed, according to the publishers, for 
adolescents from educationally disadvantaged backgrovinds (Karlstii, Madden, & 
Gardner, I97l)» 

The three remaining measures are all nvonverbal subtests drawn from a 
battery with known factorial properties (French, Ekstrom, & Price, 1969). 

These consist of a conventional test of Figure Analogies, a maze-following 
type of measure designated as Choose-A-Path and an associative (rote) memory 
test designated as Pictvire-Ntmiber. The three mesisures have also been utilized 
previovisly, as part of a guidance and reseeurch program with junior college 
populations (College Entrance Examination Board, 1969) and with Uth and 12th 
grade inner-city students (Flavigher, 197l). 



C. Criterion measures . For purposes of determining the predictive 
validity of all the above paper and pencil measures, scores were obtained 
fran questionnaires developed in a previo\as study of criteria for youth -vfork 
training programs (Freeberg & Reilly, I97l)» Based on factors obtained in 
that study, sets of variables v^ere scored to yield; (a) three coherent and 
relatively independent criterion scales for use at program ccsnpletion, (b) 
another set of three at post-program periods for those former enrollees who 
had obtained employment, and (c) a third set of three criterion scales for 
former enrollees who had not obtained employment over the defined post-program 
period. 

. Program Completion Criterion Scales are designated as; (l) "Train- 
ing Program Adjxistment" — which combines variables of trouble vfith 
police., covinselor proficiency ratings, work supervisor proficiency 
ratings, worksite absences, nvanber of jobs in vfhich the enrollee is 
willing to take training; (2) "Social A(^ustment" — combining variables 
of problems vfith people in the commvinity, family adj\astment, number 
of police contacts, peer rating score; and (5) "Job Aspiration 
Level" — based on variables of enrollee salary expectations, level 
of a highly desired job, level of long-range plans. 

. Post-Program Criterion Scales consist of one set of scales for those 
vfho had obtained full-time employment (over a period of one week or 
more) after leaving the program. These are defined as; (l) "General 
Job Success and Adjustment" — \ising variables of employer proficiency 
rating, financial savings, whether or not employed at the time of 
the interview, length of stay on job, family adj\astment, police 
contacts; (2) "Striving for Vocational Success" — which combines 
variables of job promotion, amount of first raise, problems with 



people in the community, queility of next job desired, ease with which 
contacted by the interviewer; and (3) "Stability-Mobility” — based on 
combining variables of number of places intervieiired for jobs, nimiber 
of jobs held, manber of visits to State Employment and number of 
places lived. 

Applicable post-program sceiles, for those who had not obtained 
employment, were derived from relatively weak criter3,on factors that 
were found to be neither as coherent nor as interpretable as the sets 
of factors for the other groups (Freeberg & Reilly, I97l)« These are: 
(l) "community Adj\istment" — which combines variables of trouble with 
police, willingness to contribute financially to family, ease of ob- 
taining credit; ( 2 ) "striving for Personal Improvement" — with variables 
of part-time work experience, salary expectations, ease with which 
contacted by interviewer; and (3) "Reeilistic Aspirations" — combining 
variables of estimated ability to perform on a sought-after job, 
knowledge of the job sought, salary expectations, and family adjxistment 

Sample and Data Collection 

Twenty-nine males and 4 5 females enrolled in two Neighborhood Youth Corps 
Out-of-School projects, located in a northeastern urban area, constituted the 
sample from v/hich the test data vrere obtained. The sample of were predomi- 
nantly Black high school dropouts, between l4 and 17 years of age, who had been 
eni oiled for approximately several weeks, or less, at the time they were tested 
by the project guidance counselors. Of the ih, a group of 20 who remained 
enrolled for approximately 5 to 6 months or more (arbitrarily defined as program 
"completion") could be contacted at the time they left the program and were also 
willing to respond to an individually administered program-completion criterion 
questionnaire. 



Forty-four of the 7^ enrollees were located approximately six months 
after they had left the program and consented to respond to the individvially 
administered post-program criterion questionnaire. Seventeen of these 44 
fomer enrollees obtained full-time employment (Post-Program: Employed Sample) 
and 27 had failed to obtain full-time jobs loriting for a period of one vreek 
or more (Post -Program: Not Employed Sample). Enrollees were paid for respond- 
ing to the criterion quer.tionnaires . 

Data Analyses 

Levels of concurrent’ validity for the Enrollee Test Battex-y vrere deter- 
mined from intercorrelations between the 17 tests of that battery and the six 
conventional cognitive skill measures, as obtained from a 2? x 23 inter- 
correlation matrix of all test scores. That matrix was also factor analyzed, 
as a way of summarizing overall patterns of relationships, using a principal 
axis solution with Varimax rotation of the factors to orthogonality (Kaiser, 

1958). 

Predictive validity for the 23 tests was examined by correlating each 
of the 23 test scores with factor scores derived from the applicable question- 
naire criterion scales for each of the three criterion samples — i.e.. Program 
Completion, Post-Program (Employed), and Post-Program (Not Employed). Each 
factor score consisted of a weighted sum of the standardized variables with 
loadings greater than .30. In all cases the wei^ts used were the factor 
loadings derived in a previous analysis of the questionnaire with a larger, 
more representative sample (Freeberg & Reilly, I97l) than the present one. 

Since missing data occurred throughout the samples for the test scores 
(i.e., N's ranged from 67 to 74), computation of the intercorrelation matrix 



required the vise of miSs.vag data analyses. For Vne much smaller available 
samples, used to deterioine predictive validities, missing data resulted in 
varied sample sizes (N's ranged from 10 to 2?). Where differing N's occur, 
they are appropriately indicated in the tabled results. 

Results and Discussion 

Concurrent Validation 

The matrix showing the intercorrelations for the 25 tests is presented 
in Table 1. 

Insert Table 1 about here 

Moderate to substantial levels of concurrent validity occur most con- 
sistently between fovtr subtests of the ETB and five of the six conventional 
measures of cognitive skill. These are fovind, appropriately enough, for the 
four subtests of the ETB that possess an obvious requirement mr veasoni?>s 
or verbal-intellectual capabilities (i.e.. Job Knowledge, two Practical 
Reasoning subtests and the Job Seeking Skills test). Within that set of 
coefficients--bounded by heavy lines in the upper-ri^t portion o/ the 
matrix--it can be seen that the Job Seeking Skills measui'e resvQts in the 
highest levels of concurrent validity overall, especially with the more 
complex conceptual skills involving word meaning (r = .57)^ arithmetic 
(r = .58) and paragraph comprehension (r = .75). 

It should be noted that, in general, the levels of intercorrelation among 
the six conventional measures (variables #l8 to # 25 ) are no higher than their 
correlations with the four cognitive skill measures of the ETB; although 



are eon.<?i derahly different in design-aP.nd comparatively-mT ch. shorl-ey 



in length. 

One striking featiire of the relationships between the cognitive skill 
measures is seen in the correlations of low magnitude between the Pictiire- 
Number Test and the other five conventional measures, as well as with the 
four cognitive skills subtests of the ETB.(r's ranging from .05 to .53 v^ith 
only three of the nine correlations reaching significance). Those uniformly 
low correlations for this sample of minority-group adolescents reflect a 
degree of relative independence for the purely associative, rote-memory task. 
The finding is somewhat consonant with differences in learning strategies 
that have been suggested for the acquisition of rote learning, in contrast 
to strategies believed to underlie higher-level conceptual abilities 
(Rohvrer, 197l). Moreover, when rote-memory test scores are correlated with 
measures of conceptual skills for different ethnic groups, the lovrest of 
the correlations (i.e., the greater relative uniqueness of rote memory skill) 
have been found for Black adolescents from inner-city schools (Flaugher & 
Rock, 1972 ). 

Significant correlations between the attitudinal scales of the battery 
and the cognitive-intellectual measures (Table l) are relatively few and 
scattered with no clear pattern evident. Only the several positive r's in 
the .20’s and .50’s appearing among the Job Holding Skills, Attitude Toward 
Authority and Motivation for Vocational Achievement scales suggest possible 
consistencies of these attitudes in relation to cognitive skills. For the 
most part, hovrever, those resiilts approximate the extent of relationships 
expected when attitudinal and cognitive-intellectual measures are inter- 



correlated. 
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A much broader perspective of the correlational patterns is provided 
by a factor analysis of the matrix of Table 1, which also serves as a form 
of construct validation for all of the measures vised. Loadings on the four 
factors extracted by that analysis (Table 2) confirm, in a more coherent way, 
much of what has already been discussed on the basis of visual examination 
of the individual correlations. 

Insert Table 2 about here 

Thus, Factor I, Cognitive Ability , emerges as one that is clearly 
dominated by high loadings on all of the cognitive skills measures with the 
exception of the Picture-Number Test, which shows relatively little presence 
on that dimension. Of the loadings for the noncognitive (attitude or interest) 
scales only the Attitude Toward Authority scale reaches an interpretable level 
(i.e., . 30 ), suggesting that there is a tendency for those who express greater 
acceptance of conventional authority to achieve better intellectual skills 
scores. 

Factor II represents an unquestioned dimension of generalized Interest 
in Vocational Tasks — of all sorts. Coherence of this factor is consistent 
with the findings for similar job-task interest scales using a wider range 
of ethnic groups and SES levels in the samples tested (Knapp, Grant, & Demos, 
1966). Equally consistent with previous findings is the relative independence 
of the interest factor in relation to attitudinal and intellectual skills 
measures (Cronbach, 196O; Tyler, 1956 ). 

The attitudinal scales that comprise Factor III are reasonably inter- 
pre table as a dimension of Positive Social Attitudes , with an apparent 
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vocational focus. Thus, those enrollees v^ho possess stronger feelings of Self- 
Esteem are the ones who also score higher on the Motivation for Vocational 
Achievement scale, indicate a higher degree of awareness of proper on-the-job 
behaviors (job Holding Skills) and tend to prefer Senrice job tasks found in 
occupations that entail dealing with, or helping, others (e.g., social worker, 
store clerk). These attitudinal scales are relatively independent of any 
intellectual skills measures with the exception of the positive loading found 
for the Arithmetic Test (.57)» 

The last factor extracted. Factor IV, is interpreted most sensibly as 
one that differentiates Masculine Vs. Feminine Interests and is a likely 
result of having combined males and females into a single sample. Aspects 
of the loading pattern that support the interpretation stem from the domi- 
nant positive loadings on Technical and Outdoor Vocational task interest — 
knov/n to result in higher scores for males (Freeberg, 1968) — accompanied by 
negative loadings on the more female-oriented Clerical job tasks. At the 
same time, the factor depicts an enrollee who performs more poorly in para- 
graph comprehension and also shows less v^illingness to defer present grata fi- 

2 

cation for future gain. There is also a tendency for the individual v^ho is 
on the masculine-interests side of this dimension to perform better on the 
nonverbal Choose-A-Path test. 



Female school dropouts from urban aresis who enter youth-work training 
programs are generally found to be superior to males in reading ability and 
also tend to score higher on this particular Deferred Gratification Scale 
( Freeberg, 1968 ) . 



Predictive Validation 
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Predictive validities of the 25 test scores are presented in Tables 
5, 4, and 5 for the Prograin Completion, Post-Pro^am (Employed), and Post- 
Program (Not Employed) criterion samples respectively. 

Insert Tables 5, *<•, and 5 about here 

In what must necessarily be a very cautious and broad interpretation 
of these small-sample resiilts, it is apparent that the most consistent 
patterns and highest levels of significant validity are dominated by the 
cognitive-intellectual tests. Few scattered validities for the interest or 
attitudinal scales appear across the three criterion samples, although those 
fev 7 significant ones serve as an aid in interpreting the overall findings. 
Specifically, among the validities for the Program Completion criteria 
(Table 5), "Social Adjustment" (with regard to family, peers, police, other 
members of the community) is best predicted from the higher level conceptual 
skills of Arithmetic (r = .66), Paragraph Meaning (r = .52), and Practical 
Reasoning: Zip Coding (r = .51)* That criterion dimension is eilso associated 
positively with Self-Esteem scores (r = .47) and negatively (r = -.44) with 
the level of interest shown for Outdoor Job Tasks (i.e., lower level, farm- 
related tasks requiring physical labor). 

The other set of validities with some degree of consistency in their 
pattern of significance is, again, found to occur for a nvmiber of cognitive 
skills tests. These appear for the Post-Program (Employed) enrollee sample 
as predictive of the "Job Success and Adjustment" criterion scale (Table 4). 
In this instance, significant r's occur over a fairly wide range of cognitive 
skills encompassing the purely nonverbal Figure Analogies (r = .57) and 
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Picttire Number ^sts (r = .5l)j as well the_ more verball?/’ . demandinc 
Vocabulary (r = .49) , Aritlmietic (r = .51) and Job Seeking Skill tests 
(r = .51), v^ith the Practical Reasoning: Zip Code and Paragraph Meaning 
tests falling just short of the .05 significance level. 

In contrast to the above findings, an unusual trend in significant 
negative relationships results for two of the nonverbal cognitive skills 
tests. The finding might easily be written off as aberrant, by virtue of 
the small N's, if it did not persist as a similjir pattern in both the 
Program-Completion and Post-Program (Not Employed) samples (tvro groups vfhich, 
it should be noted, are conceptually more similar to one another than they 
are to a Post-Program group of enrollees who obtained full-time employment). 
For those tv.'o samples. Choose -A-Path scores are negatively related to the 
"Training Program Adj vis tment" scale at program completion (r = .54; Table 5) 
and to "General Community Adjustment" in the Post-Program period (for ex- 
enrollees Wot Employed, r = -.59; Table 5). The Picture-Number test results, 
in a similar pattern of negative correlations for those same criterion 
samples and scales — although barely short of significance in the case of 
the Program Completion sanple (r = -.4l; p = .06). Add to this on an over- 
all pattern in the direction of negative validities for Technical and Outdoor 
interest scales with these same criteria and one tenuous clue to a sensible 
explanation of these results is possible. It is simply that an underlying 
sex differentiation vras obscured by the necessity for combining male and 
female groups . 

In essence, since both the Technical and Outdoor task preferences are 
known to be distinctly greater for the male enrollees, it is possible to 
assume that the remainder of this validity pattern is also sex dependent — i.e 
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^ that better -^r,£orjnance foaijnai-es-'^n t3aese-4»ro nonverbal^cDgaii-i.-v: tests-*.- • 

associated with poorer "Training Program Adjustment" (at the end of the 
program) and poorer "General Community Adj\istment" (post-program) for those 
meiles who do not obtain employment. In partial support of this interpre- 
tation, it should be recalled that Choose-A-Path appeared as a masculine- 
orienbed measure on the Masculine Vs. Feminine Interests factor. 

One other point that should be noted in Tables 5, and 5 is that, 
..despite the relative independence of the Pictiire-Nvimber test in relation 
to other cognitive skills , the measiire tended to display significant (or near 
significant) predictive validities across the three criterion samples. 

Similar hints of the value of rote memory skill have appeared in its relation 
to job performance criteria with samples of Black Civil Service employees, 
as contrasted with lesser validity for liHiite employees (Campbell, 1972). 

Few of the remaining predictive validities are worth specific interpreta- 
tion in such a small-scale stvidy phase, except for mention of several that 
appear with the "Job Stability - Mobility Criterion" scale (Table 4). Among 
these ex-enrollees (Qnployed), higher scores on the Motivation for Vocational 
Achievanent scale were associated with significantly greater mobility (in 
jobs, residence, and degree of job search effort; r = .5^) along with a ten- 
dency— although not quite significant- -for the more mobile enrollees to 
express lesser willingness to defer gratification (r = .45; p < .lO). In 
addition, the more mobile the former enrollee the poorer his performance on 
the Arithmetic Skills test (r = -.55)* 

"Bnployment" as Criterion 

In the above analyses it was necessary and logical to deal with differ- 
ent post-program criterion measures on the basis of whether former enrollees 

er|c 
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did ndtr- oB‘oail'r"Snployinent inis divTsion, however, precluded the 
opportunity to examine the predictive value of the tests for what would 
normally be considered a major criterion measure applicable to any youth- 
work training program— i.e . , Employment Vs. No Employment. By combining 
the two post-program samples into a single sample of former enrollees it 
is possible to compute the biserial correlations between each test and the 
dichotomo\as employment criterion as shown in Table 6. 



Insert Table 6 about here 

None of the measures are found to be correlated significantly with 
employment as a criterion for this sample, nor are any trends discernible. 
The negative findings not only stand by themselves, they also buttress 
argimients against total dependence on a single, all-encompassing, program 
criterion of employment (as important as it may be). Limiting test valida- 
tion to such a single outcome measure would have failed to point up clues 
to the predictive potential of the measures, as found here, when a wide 
variety of post-program performance capabilities are \ised to define end-of- 
program or post-program "success.” 

Concl\asions 



Measures of the Enrollee Test Battery, designed specifically for use 
with disadvantaged adolescents, appear fairly coherent and logical in their 

patterns of relationships \irith each other and with conventional cognitive 
skills tests. Substantial levels of concurrent validity for four of the 
subtests in the ETB — each of which require knowledge or reasoning capa- 
bility-confirm their intended cognitive dimensional quality. At the same 
time, noncognitive attitude and interest scales yield factors that are 
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interpretable and relatively independent of cognitive skills 4 Demonstration 
of the relative uniqueness of such dimensions is, of coiirse, desirable for 
measurement purposes (i.e., serving as a form of construct validity) and is 
believed most readily obtainable, for an educationally disadvantaged group with 
poor reading ability, when verbal skill (i.e., reading) requirements are 
removed as a primary component of test response. 

The more critical questions of test value, however, can only be approached 
on the basis of their correlations with a broader range of performance criteria, 
and in a predictive framework. Within the limitations of very smeill and re- 
stricted samples, the results of these preliminary aneilyses have suggested a 
dominant predictive role for the various cognitive skills measures with 
severel criterion dimensions. Special interest was focused on the rote 
memory measure because of its predictive possibilities in spite of its unique 
standing (i.e., low correlations) with the other cognitive skills tests. 
Unfortunately, a number of the cognitive test validities and their interpre- 
tations were believed to be confounded, in important ways, by having combined 
males and females in the same sample. The obvious need in future research 
is to obtain follow-up samples of sizes that are sufficient to allow for 
separate analyses by sex (recognizing the cost and difficulties in doing so 
when the respondents are minority group, school dropouts, residing in poverty 
level urban areas). 

Patterns of usable predictive validities for attitude and interest scales 
might be inferred from the resTilts (e.g., for Technical and Outdoor interest, 
Self-Esteem, Motivation for Vocational Achievanent). But, for the most part, 
those findings should be considered inconcltisive without larger, more repre- 
sentative, longitudinal samples and more definitive analyses by sex. 
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Part of any cbntintie'a' 'develBpment of measures in the Enrollee Test 
Battery must also include refinement of the criterion scales (e.g.^ based 
on criterion relevance or equivalence), as well as the derivation of useful 
scoring profiles, the weighting of tests for optimum prediction of multiple 
criteria, and the development of normative data — all of v^hich are to be • 
incorporated in study efforts presently being planned or in progress. 







SO 
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Table 3 

Predictive Validities for 23 Tests with Three 
Program-Completion Criterion Scales 

Criterion Scales 



Tests 


Training Program 
Ad.iustment 


Social 

Ad.iustment 


Job Aspiration 
Level 




(N = 20) 


(N = 20) 


(H = 19) 



1. 


Job Knowledge 


-.23 


.19 


.28 


2. 


Practical Reasoning 


.21 


.51* 


-.02 




Zip Coding 






3. 


Practical Reasoning 


.54 


.54 


-.14 




Map Reading 








k. 


Job Seeking Skills 


-.23 


.16 


.02 


5. 


! Clerical 


.19 


.28 


-.20 


6. 


1 Service 


.30 


.04 


-.19 


7. 


\ Technical 


-.38 


-.18 


.14 


8. 


Voc. J ^ ,, 

Interests < Outdoor 


-.41 (p = . 06 ) 


-.44 


.08 


9. 


) Science 


.07 


.14 


.05 


10. 


/ Business 


-.10 


.01 


.07 


11. 


\ Aesthetic 


-.01 


-.20 


.02 


12. 


Voc. Aspirations [■linus 


.22 


.16 


-.51 




Voc. Plans 








13. 


Attitude Towards Authority 


.16 


.31 


.10 


Ih. 


Self-Esteem 


-.14 


.47* 


.28 


15. 


Deferred Gratification 


-.02 


-.11 


-.09 


16 . 


Motivation for Achievement 


-.29 


.12 


-.14 


17. 


Job Holding Skills 


.15 


.25 


.04 


18 . 


ABLE Vocabular:'' 


-.05 


.09 


-.17 


19. 


ABIjE Arithmetic 


.09 


.66* 


.26 


20. 


Stanford Achievement PM 


.08 


.52* 


.05 


21. 


Choose-A-Path 


-.54* 


-.09 


.01 


22. 


Figure Analogies 


-.04 


-.10 


.14 


23. 


Picture Number 


-.41 (p = . 06 ) 


.07 


.25 



*Significant at .05 level. 
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Table k 

Predictive Validities for 23 Tests with Three Post- 



Program Criterion ScaJ.es (Employed Sample) 






Criterion Scales 




Job Success 
Tests & Ad.iustment 


Striving for 
Vocational Success 


Job Stability- 
Mobility 


(N = 15) 


(N = 10) 


(N = 15) 



1. 


Job FCnowledge 


.OU 


.04 


.16 


2. 


Practical Reasoning 


.46 


-.51 


-.07 




Zip Coding 








3. 


Practical Reasoning 


.18 


-.57 


-.12 




Map Reading 








h. 


Job Seeking Skills 


.51* 


-.32 


.18 


5. 


/ Clerical 


.09 


.36 


-.28 


6. 


1 Service 


-.27 


-.02 


.39 


7. 


\ Technical 


-•25 


.12 


-.04 


8. 


Interests { 


-.36 


.05 


.06 


9. 


1 Science 


.09 


.21 


-.02 


10. 


I Business 


- . 39 


.18 


-.51 


11. 


\ Aesthetic 


-.28 


.06 


-.12 


12. 


Voc. Aspirations Minus 


-.l6 


-.58 


.10 




Voc . Plans 








13. 


Attitude Towards Authority 


.15 


- .86* 


•25 


ll». 


Self-Esteem 


-.30 


-.05 


.07 


15. 


Deferred Gratification 


.26 


-.38 


-.43 


16 . 


Motivation for Achievement 


-.17 


-.18 


.54* 


17. 


Job Holding Skills 


.26 


•15 


-.06 


18 . 


ABLE Vocabvilary 


. 49*^ 


1 

ro 


-.27^ 


19. 


ABLE Arithmetic 


.51*^ 


.ir 


-.55*®' 


20. 


Stanford Achievement PM 


.43"" 


-. 26 ^ 


-.12®" 


21. 


Choose-A-Path 


.Ol®’ 


-.11^ 


-.17^ 


22. 


Figure Analogies 


.57*®' 


-.39^ 


-.25^ 


23. 


Picture Number 


.51*®" 


-.08^ 


-. 18 ®" 



= 17; \ = 11. 



I ^Significant at .05 level. 
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Table 5 

Predictive Validities for 23 Tests with Three Post-Program 
Criterion Scales (Mot-Employed Sample) 





Tests 


General Social 
Ad,iustment 


Criterion Scales 

Striving for 
Personal Improvement 


Realistic 

Aspiration 


1. 


Job Knowledge 


(N = 27) 
-.03 


(N =17) 
.10 


(N = 27 ) 
.01 


2. 


Practical Reasoning 


.Ok®' 


-.12 


.15^ 


3. 


Zip Coding 
Practical Reasoning 


-.1^ 


-.54 


.09®' 


It. 


Map Reading 
Job Seeking Skills 


-.12^ 


.27 


-.19^ 


5. 




f Clerical 


.09 


.21* * 


.10 


6. 




Service 


-.31 


-.19 


-.31 


7. 




i Technical 


-.lt5* 


-.12 


-.05 


8. 


Voc. 


/ Outdoor 


-.36 


-.12 


-.05 


9. 


Interests 


1 Science 


-.26 


-.21* 


-.05 


10. 




[ Business 


-.25 


.17 


-.06 


11. 




1 Aesthetic 


-.29 


.22 


-.03 


12. 


Voc. Aspirations Minus 


. .00 


-.58 


.16 


13. 


Voc. Plans 

Attitude Towards Authority 


-.19 


-.1*2 


-.29 


lit. 


Self-Esteem 


-.06 


-•IT 


-.32 


15. 


Deferred Gratification 


.29 


-.22 


-.03 


16 . 


Motivation 


for Achievement 


.00 


-.30 


.11* 


17. 


Job Holding Skills 


-.11^ 


-.13 


-.05^ 


18 . 


ABLE Vocabulary 


-.20^ 


.•03 


1 

b 


19. 


ABLE Arithmetic 


-. 16 ^ 


-.09 


-. 10 ° 


20. 


Stanford Achievement PM 


.12^^ 


-.13*^ 


.28" 


21. 


Choose-A-Path 


-.39*^ 


CO 

0 

r 


-.21*^ 


22. 


Figure Analogies 


. 06 ^ 


-.1*8**^ 


.17^^ 


23. 


Pictvu'e Number 


-.39*^^ 


.02^^ 


-. 06 ^' 



% = 26 ; = 25; =2lt; % = 15. 

*Significant at .05 level. 
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Table 6 

Predictive Validities of 23 Tests Using Employment Vs. 
No Employment as Criterion 
(H = 1*1*) 





Test 




bis . 




Test 


^bis 


1. 


Job Knowledge 


.09 


13. 


Attitude Towards Authority 


.26 


2. 


Practical Reasoning 


-.15 


ll*. 


Self-Esteem 


.12 




Zip Coding 






15. 


Deferred Gratification 


.10 


3. 


Practical Reasoning 


.19 










Map Reading 




l6. 


Motivation for Achievement 


-.25 


1*. 


Job Seeking Skills 


.11 


17. 


Job Holding Skills 


-.07 


5. 




Clerical 


.26 


18 . 


ABLE - Vocabuleiry 


.16 


6 . 




Service 


.01* 


19. 


ABLE - Arithmetic 


.01 


7. 


Voc. 


1 Technical 


.08 


20. 


Stanford Achievement 


.19 


8. 


Interests < 


y Outdoor 


.22 


21. 


Choose-A-Path 


.28 


9. 




1 Science 


.05 


22. 


Figure Analogies 


.11* 


10. 




Business 


.11* 


23. 


Picture Niunber 


-.22 


11. 




1 Aesthetic 


.11 








12. 


Voc. Aspirations Minus 


.00 









I r|c ]. 28 



